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ABSTRACT 

A  theory  for  neuron  selectivity  in  visual  cortex  is  presented  and 
compared  with  neurophysiological  data.  In  this  theory*  a  neuron  becomes 
maximally  selective  over  the  set  of  patterns  presented  to  it.  Theoretical 
predictions  correspond  well  with  results  of  various  experimental  paradigms 
including  normal  rearing*  monocular  and  binocular  deprivation,  and  reverse 
suture.  Extensions  of  the  theory  to  cortical  networks  based  on  anatomical 
data  are  discussed.  The  ideal  synapse*  which  can  alter  its  function  between 
excitatory  and  inhibitory  modes*  must  be  resolved  into  excitatory  and 
inhibitory  synapses  to  agree  with  anatomical  observation.  A  mathematical 


appendix  provides  stability  analysis  of  the  model's  fixed  point  behavior. 
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INTRODUCTION 


Several  models  for  postnatal  development  in  area  17  of  visual  cortex  have 
been  put  forward  previously  (e.g.  von  der  Malsberg,  1973  ;  Perez  et  al.,  197b  ; 
Cooper  et  al.,  1979).  The  theory  presented  here  describes  not  only  the 
selective  tuning  of  neurons  in  this  area  to  specific  visual  stimuli*  but  also 
tne  interaction  of  visual  signals  from  both  eyes.  Analytical  studies  and 
computer  simulation  indicate  that  response  characteristics  of  model  neurons 
evolve  as  a  function  of  the  visual  environment  in  a  manner  consistent  with 
experimental  studies  from  several  laboratories. 

In  the  formulation  of  theories  describing  neural  processing  of  any  kind, 
certain  common  elements  must  be  considered,  for  example  the  connectivity  of 
the  neuronal  network.  Consider  a  network  (fig.  1)  in  which  one  population  of 
neurons  d  projects  to  another  population  £..  The  net  connectivity  between 
the  j-th  cell  of  the  input  population  (dj)  and  the  i-th  output  cell  (cj) 
is  given  by  the  value  j .  a  typical  connectivity,  M^j,  represents  the 
net  influence  of  several  synapses,  both  in  parallel  and  in  series,  both 
excitatory  and  inhibitory,  possibly  involving  interneurons.  M^j  represents 
the  efficacy  of  an  'ideal  synapse'  (Nass  and  Cooper,  1975),  a  useful  concept 
to  the  mathematician,  but  one  which  falls  short  of  ideal  in  the  anatomist's 
eyes.  The  idea  is  that  each  input  axon  firing  frequency  is  converted  to  some 
net  postsynaptic  membrane  depolarization  in  a  given  cell  in  the  target 
population  (c),  and  eventually  will  influence  the  firing  of  the  cell. 


r  t. 


3 


abinJC 


f 


2 


Mathematically  speaking,  the  synaptic  array  M  operates  on  the  stimulus 
vector  d  producing  the  response  vector  c  (Eqn  1). 


c  =  Md 


[1] 


Similar  assumptions  have  been  described  in  the  two  previous  talks 
(Anderson  ;  Kohonen).  In  effect  they  have  said  that  the  connectivity  matrix  M 
can  evolve  to  function  as  a  memory.  By  'memory'  we  generally  mean  that  the 
synaptic  strengths  M^j  contain  information  in  that  they  yield  'appropriate 
responses'  in  the  postsynaptic  population  c  to  certain  patterns  of  afferent 
activity.  Rigorous  mathematical  theorems  can  be  proven  to  show  that  such 
memory  systems  work.  Kohonen  et  al  (19771  used  facial  photographs  encoded 
into  a  string  of  darkness  (brightness)  values  each  representing  a  small 
element  of  the  visual  field.  The  coded  information  from  each  pattern  is 
distributed  over  all  synaptic  iunctions  in  the  network.  This  example  is  a 
beautifully  graphic  demonstration  of  what  is  meant  by  a  distributed  memory. 
This  idea  has  the  important  quality  that  if  any  small  portion  of  these 
junctions  is  lost,  no  specific  faces  are  lost  -  rather  the  signal-to-noise 
ratio  is  reduced. 

The  principles  which  determine  the  evolution  of  a  synaptic  network  with 
experience  have  to  be  formulated  carefully  to  yield  a  system  which  is  capable 
of  both  extracting  information  from  the  environment  and  accessing  that  same 
information  at  a  later  time.  Ideally,  the  postsynaptic  population  c  will 
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learn  to  respond  appropriately  to  inputs  d  depending  upon  whether  or  not 
similar  patterns  have  already  been  presented  and,  if  so,  in  what  context. 
Various  types  of  modification  rules  yield  memories  which  are  suited  to  a 
variety  of  tasks.  Over  the  last  several  years  our  group  has  developed  a  model 
of  binocular  orientation-selective  cells  in  visual  cortex  by  refining  a  set  of 
equations  for  synaptic  plasticity  (Mass  and  Cooper,  1975;  Cooper  et  al,  1979; 
Bienenstock  et  al,  1982)  and  is  based  on  a  diverse  collection  of  physiological 


data 
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REVIEW  OF  RELEVANT  EXPERIMENTAL  DATA 


Since  the  original  work  of  Hubei  and  Wiesel  (Hubei  and  Wiesel.  1959)  that 
led  to  the  realization  that  visual  cortical  cells  respond  in  a  highly  specific 
fashion  to  visual  stimuli.  These  studies  have  been  extended  to  the 
development  of  this  specificity  in  the  newborn  animal.  Michel  Imbert  and  his 
collaborators  (Imbert  and  Buisseret*  1975)  have  classified  visually  responsive 


cells  into  three  groups  -  aspecific,  immature  and  specific  -  according  to 

their  response  properties  and  receptive  field  arrangements  (figure  2).  \ 

1° 

Aspecific  cells  are  characterized  by  rather  indiscriminant  responses  to 
circular  stimuli  moving  in  any  direction  across  their  receptive  fields.  The 
receptive  field  is  usually  rather  large  in  size  and  circular  in  shape. 

Immature  cells  are  more  highly  tuned  in  their  response:  they 
characteristically  respond  best  to  a  correctly  oriented  rectilinear  stimulus 
moving  across  the  receptive  field*  and  there  is  always  an  orientation  for  the 
stimulus  that  will  evoke  a  negligible  response.  Their  receptive  fields  are 
large*  but  more  rectangular  than  the  receptive  fields  of  the  aspecific  cells. 
The  most  discriminant  of  all  cells*  the  specific  cells*  show  a  sharp  response 


5 


to  an  optimal  orientation  of  the  stimulus  and  are  much  less  responsive  to 
other  orientations.  These  cells  have  receptive  fields  that  are  smaller  in 
size  than  those  of  the  two  previous  cell  classes,  and  are  rectangular  in 
shape. 


The  evolution  of  these  different  cell  types  has  been  studied  for  kittens 

raised  in  different  visual  environments  (Fregnac  and  Imbert,  1977,  1978; 

Fregnac,  1978).  The  results  for  kittens  raised  in  a  normal  visual  environment 

(NR)  and  for  those  raised  in  complete  darkness  (DR)  since  the  first  or  second o  " 

|  Jhy»e&  j, 

day  of  age  are  compared  in  figures  3  and  4.  This  data  indicates  that  in  the  |  Hot  t  4k 


earliest  stages  of  development  there  are  some  cells  present  that  exhibit 
specific  cell  properties.  However,  visual  experience  is  critical  in  the 
development  of  these  cells.  The  normally  reared  animals  show  a  pronounced 
increase  in  the  number  of  specific  cells  relative  to  aspecific  cells,  while 
dark  reared  animals  show  just  the  opposite.  The  high  degree  of  malleability 
of  cortical  cells  in  this  critical  period  from  about  day  17  to  day  70  is 
illustrated  in  figure  5.  We  see  that  as  little  as  six  hours  of  visual 
experience  at  42  days  of  age  can  dramatically  alter  the  ratio  of  specific  to 
aspecitic  and  immature  cells  (Buisseret  et  al.,  1978). 


T~^yoct. 

uWrWu 


In  addition,  ocular  dominance  appears  to  depend  on  visual  experience.  It 
is  found  that  before  the  age  of  21  days,  independent  of  visual  experience, 
most  immature  and  specific  cells  are  driven  optimally  by  the  eye  contralateral 
to  the  hemisphere  in  which  they  are  located.  After  three  weeks  of  age, 
normally  reared  animals  exhibit  a  significant  increase  in  binocularly  driven 
cells  (Fregnac  and  Imbert,  1978).  This  result  is  also  found  for  animals  that 
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are  dark  reared  (Blakemore  and  Mitchell,  1973;  Imbert  and  Buisseret,  1975; 
Blakemore  and  Van  Sluyters,  1975;  Buisseret  and  Imbert,  1976;  Leventhal  and 
Hirsch,  1980;  Fregnac  and  Imbert,  1978).  However,  in  animals  whose  eyelids 
have  been  sutured  at  birth  (and  thus  deprived  of  binocular  pattern  vision, 

BD),  a  higher  proportion  of  selective  cells  are  found  compared  to  dark  reared 
animals  and  the  proportion  of  binocular  cells  is  less  than  normal  (Wiesel  and 
Hubei,  1965;  Blakemore  and  Van  Sluyters,  1975;  Kratz  and  Spear,  1976; 

Leventhal  and  Hirsch,  1977;  Watkins  et  al.,  1978). 

Finally,  one  of  the  clearest  examples  of  the  dependence  of  cortical 
development  on  the  environment  occurs  when  an  animal  is  raised  with  one  eye  in 
competitive  advantage  over  the  other.  When  monocular  lid  suture  (MD)  is 
performed  during  the  critical  period,  a  rapid  loss  of  binocularity  to  the 
profit  of  the  open  eye  occurs  (Wiesel  and  Hubei,  1963,  1965).  If  the  sutures 
are  reversed  (RS),  the  closed  eye  opened  and  the  experienced  eye  closed,  it  is 
possible  to  observe  a  complete  reversal  of  ocular  dominance  (Blakemore  and  Van 
Sluyters,  1974). 


FORMULATION  OF  THE  THEORY 


The  theory  we  now  describe  deals  with  the  development  of  selectivity  ana 
ocular  dominance.  The  pattern  environment  available  to  the  neuron  consists  of 
impulses  directly  associated  with  a  specific  receptive  field  on  the  retina 
and  secondary  lateral  influences  from  cortical  cells  having  different  primary 
receptive  fields  which  may  overlap  to  varying  degrees.  One  can  easily 
imagine  how  synaptic  delays  can  account  for  the  translation  of  a  spatio- 
temporal  pattern  such  as  a  moving  bar  to  a  near- simultaneous,  purely  spatial 
pattern  of  postsynaptic  potentials  at  a  neuron's  dendritic  surface.  Hence, 
the  notion  of  a  visual  stimulus  should  be  considered  in  its  most  general 
sense. 

Refraining  from  choosing  completely  hoc  assumptions,  we  first 
consider  what  aspects  of  a  given  synapse's  environment  might  influence  its 
development.  Changes  in  the  physical  structure  of  either  the  presynaptic  or 
postsynaptic  elements  or  the  cleft  itself,  as  well  as  the  chemistry  of  the 
synapse’s  immediate  neighborhood  come  to  mind  as  primary  candidates.  Of 
course  consideration  of  proposed  biochemical  mechanisms  (e.g.  Changeaux  et  al, 
1973;  can  be  most  useful  at  this  stage  of  model  development. 

Let  us  begin  developing  our  model  with  a  formal  statement  of  the 
functional  dependence  of  dm/dt  (Eqn  2).  Namely,  we  assume  here  that  the 
change  in  efficacy  of  a  given  synapse  depends  on  the  efficacy  itself,  both 


o 


the  presynaptic  and  postsynaptic  activities*  as  well  as  a  time  average  of  the 
postsynaptic  actvity.  Global  factors  may  well  play  a  strong  modulatory  role 
(see  for  example  Kasamatsu  and  Pettigrew)*  but  we  do  not  consider  them  here. 


m  =  m(m,d,c,c) 


[2] 


Two  linear  assumptions  are  made  in  this  model.  First,  the  transfer 
function  (Eqn  3a),  or  input-output  relation,  is  simply  an  inner  product.  More 
complicated  functions  have  been  tested  and  do  not  seem  to  have  a  qualitative 
influence  on  the  results  as  long  as  they  are  monotonically  increasing  in  both 
m  and  d.  The  modification  function  (Eqn  3b)  includes  the  second  linear 
assumption,  namely  that  dm/dt  is  linear  in  d.  Neglecting  the  second  term,  we 
see  that  the  model  relies  on  the  premise  that  the  change  in  the  synapfc 
vector  is  parallel  to  the  input,  hence  the  response  characteristics  of  the 
cell  are  influenced  for  that  input  more  than  any  other  possible  stimulus  of 
the  same  magnitude. 


c 


=  ?midi 


[3a] 


m^  =  ^(c,c)d£ 


L3b] 


This  model  is  predicated  on  the  notion  that  it  ^>0  for  a  given  input  and 
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the  evoked  response  then  the  change  such  that  the  cell's  response  to  a 
future  presentation  of  the  sane  stimulus  is  enhanced.  This  idea  was  first  put 
forward  by  Hebb  (1949)  so  we  will  term  such  modification  hebbian.  Similarly 
a  stimulus-response  combination  yielding  a  negative  value  for  ^  decreases  the 
cell's  affinity  to  that  stimulus  (anti-hebbian) .  The  conditions  on  ^  are 
illustrated  in  figure  6. 

Since  we  are  modelling  cells  which  become  highly  selective,  we  design  a 
system  which  weakens  responses  to  weak  patterns  and  strengthens  responses  to 
strong  ones.  The  strength  of  a  neuron's  response  to  a  pattern  is  evaluated 
relative  to  a  time-varying  threshold  value  (8)  intrinsic  to  the  cell,  at 
which  £  vanishes.  This  'floating*  threshold  is  linked  to  the  expectation 
value  of  the  cell's  response  evaluated  with  respect  to  the  stimulus 
environment.  If  the  time  scale  for  synaptic  modification  is  sufficiently 
larger  than  the  usual  duration  of  an  input  presentation,  then  this  expectation 
value  is  close  to  a  time-average. 

If  the  threshold  could  not  vary  in  time,  the  system  would  require  some 
very  specific  assumptions  (Cooper  et  al,  1979)  in  order  to  account  for  the 
highly  tuned  cells  observed  in  visual  cortex.  By  allowing  8  to  vary  with 
the  average  response  in  a  faster-than-linear  fashion,  the  response 
characteristics  evolve  to  maximum  selectivity  over  the  input  environment. 

This  spatio-temporal  modification  scheme  forces  8  and  the  nontrivial 
response  values  towards  a  common  value  such  that  the  threshold  is  driven 
between  them.  When  this  occurs  the  response  values  below  6  decrease 
toward  zero  and  8  'overtakes'  the  other  values  which  have  begun  to 


increase.  Eventually  only  a  small  subset  of  the  stimuli  drive  the  cell  above 
the  modification  threshold.  In  certain  cases  (see  appendix)*  it  has  been 
proven  that  maximum  selectivity  is  attained.  Barring  situations  where  some 
inputs  have  negative  inner  products  (see  Bienenstock*  1980).  computer 
simulations  indicate  that  the  neuron  is  always  driven  to  a  maximally 
selective  state. 

The  simple  case  of  a  neuron  driven  by  two  equiprobable  two-dimensional 
stimuli  provides  a  good  example  for  examining  these  properties  of  the  model. 
The  input  environment  consists  of  two  linearly  independent  vectors*  d*  and 
d^,  which  are  normalized  and  have  a  nonnegative  inner  product.  They  can  be 
represented  in  a  common  vector  space  with  the  synaptic  state  vector  m*  since 
each  atferent  (a  component  in  each  d1)  corresponds  to  a  unique  ideal  synapse 
(a  component  in  m) .  We  begin  by  seeking  points  which  are  candidates  for 
stationary  final  states  (fixed  points)*  then  we  will  look  at  the  behavior  of 
the  system  relative  to  these  points.  If  we  assume  that  the  synaptic  weights 
change  slowly  relative  to  the  frequency  of  stimulus  presentation*  then  we  can 
assume  that  a  fixed  point  represents  any  state  for  which  the  net  modification 
induced  by  the  two  patterns  is  zero.  Since  the  patterns  are  independent* 
dm/dt  must  be  zero  for  each  stimulus. 

Let  9  =  c^  give  .the  threshold  value  for  the  ^  function.  We  see  then 
that  the  locus  of  points  for  which  ^=0  for  a  given  input  (fig  7)  is  the  union 
of  two  loci,  namely  a  line  corresponding  to  c=0  (Eqn  4a)  and  a  parabola 
corresponding  to  c=6  (Eqn  4b).  The  intersections  of  these  isoclines  give  the 
four  fixed  points*  at  which  both  patterns  must  give  ^=0.  We  define  the  order 
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of  a  fixed  point*  s*  to  be  the  number  (0.  1*  or  2*  in  this  example)  of 
patterns  which  yield  the  response  c  =  Figure  7  divides  the  space  into  four 
regions*  characterized  by  the  sign  of  ^  attributed  to  each  pattern.  Thus  we 
see  that  the  asymptotically  stable  final  states  are  fixed  points  maximum 
selectivity.  An  example  of  local  stability  analysis  is  given  in  the 
appendix. 


m*d^  =  0 


i  =  1,2  L4a] 


m^d1  =  (m*d)2 


i  =  1.2 


L4b] 


DEVELOPMENT  OF  NEURONS  UNDER  VARIOUS  REARING  CONDITIONS: 


COMPARISON  OF  THEORY  AND  EXPERIMENT 
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The  simulated  behavior  of  neurons  in  visual  cortex  with  binocular 
connectivity  is  illustrated  in  figure  8*  The  seemingly  inconsistent 
experimental  results  (MD  vs  BD)  are  faithfully  reproduced  by  computer 
simulation  according  to  the  model  outlined  here.  Thus  this  theory  gives  one 
explanation  which  resolves  the  paradox.  Each  of  these  paradigms  was  tested  in 
both  deterministic  and  stochastic  simulation  algorithms  over  several  pattern 
sets.  The  model  withstood  considerable  noisy  input*  indeed  successful 
simulation  of  some  paradigms  (RS  in  particular)  required  that  a  noiselike 


component  accompany  the  'pure'  inputs. 
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Binocular  interactions  do  not  play  a  special  role  in  understanding  the 
behavior  of  this  theory  with  respect  to  either  binocular  deprivation  or 
(correlated)  normal  rearing.  That  is*  due  to  idealizing  assumptions  (for 
example,  disparity  between  lett  and  right  inputs  is  nat  considered)  these 
two  cases  drive  the  model  neuron  just  as  it  would  be  driven  if  its  connections 
were  strictly  monocular.  Binocular  stimuli  presented  in  NR  simulations  were 
exactly  correlated  so  that  each  pattern  incident  to  the  lett-eye  synapses  was 
consistently  accompanied  by  a  corresponding  pattern  to  the  right-eye  synapses. 
Thus  each  binocular  input  enjoys  a  one  to  one  correspondence  with  a  particular 
input  to  either  eye.  The  left  and  right  components  of  each  pair  used  here  are 
identical,  as  a  consequence  the  cell  tunes  to  the  same  pattern  in  each  eye 
(Fig  8a).  Binocularly  deprived  input  environments  consisted  of  stimulus 
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components  uniformly  distributed  over  some  range  with  zero  mean.  In  this  case 
(BD),  the  average  response  of  the  cell  is  null  and  so  o  is  always  non¬ 
negative  resulting  in  the  synaptic  state  modifying  randomly  (Fig  8b). 

The  development  of  a  neuron  receiving  patterned  input  from  only  one  eye 
(and  uniform  noise  from  the  other)  is  interesting.  The  naive  prediction 
incorrect.  Even  though  the  response  curve  seeks  maximum  selectivity  with 
respect  to  the  open  eye.  the  response  to  the  other  eye  does  not  fluctuate 
randomly.  Rather,  as  in  the  laboratory,  the  neuron  becomes  nonresponsive  to 
inputs  to  the  deprived  eye  (Fig  8c).  Asymptotic  convergence  to  this  state  is 
assured  regardless  of  the  initial  state.  The  theoretical  implications  for 
the  RS  paradigm  are  as  follows:  A  monocularly  deprived  neuron,  having  reached 
a  monocular  selective  state  is  driven  to  another  monocular  selective  state 
preterring  the  newly  opened  eye  upon  reversal  of  suture  (Fig  8d) . 

These  monocular  results  rely  upon  some  activity,  albeit  purely  random,  to 
be  present  in  the  atferents  from  the  closed  eye.  Such  noise  may  be  due  to 
ditfuse  light  through  the  eyelid  or  spontaneous  firing  of  LGN  and/or  retinal 
neurons.  As  a  neuron  becomes  selective  with  respect  to  the  open  eye,  those 
few  patterns  which  are  preferred  give  a  response  near  threshold  whereas  the 
other  patterns  give  a  much  lower  response.  In  either  case  ^  is  near  zero. 
Noise  accompanying  a  preferred  pattern  drives  the  neuron  from  the  modification 
threshold,  so  the  deprived  synapses  grow  stronger.  However,  the  opposite 
effect  weakens  the  synapses  when  'uninteresting'  patterns  are  presented.  A 
mathematical  demonstration  of  this  argument  is  given  in  the  appendix. 


THE  MANY-NEUROW  PROBLEM 


So  far  we  have  been  discussing  selectivity  on  the  single  unit  level. 
However,  cortical  anatomy  suggests  that  the  geniculocortical  component  of  the 
atferents  to  a  cell  is  a  small  fraction  of  those  afterents  (Hornung  and  Garey* 
1981 Thus  it  is  very  likely  that  cortico-cortical  inputs  play  an  important 
role  in  the  development  of  neuron  selectivity  (Creutzfeldt  et  al.»  1974. 
Sillito.  197>).  The  development  of  selectivity  then  might  be  viewed  as  a  many- 
neuron  problem. 

Formulation  of  the  many-neuron  problem  raises  several  issues.  These  may 
be  classitied  into  three  broad  categories.  The  first  concerns  relevant 
cortical  anatomy.  Neuroanatomical  studies  have  provided  extensive  detail.  He 
would  like  to  include  details  relevant  to  the  development  of  selectivity,  yet 
keep  the  model  to  a  manageable  complexity.  Second,  the  problem  of  response 
dynamics  needs  to  be  examined.  The  prescence  of  cortico-cortical  loops  could 
lead  to  'resonance*  effects  in  the  response  of  a  cell  to  a  pattern.  Thus  the 
transfer  function  of  the  cell  population  must  be  more  caretully  defined  than 
the  single  cell  case  (see  equation  3a).  Two  possible  solutions  are  (a) 

Cortical  activity  rapidly  settles  before  the  afferent  message  changes  and  (b) 
Cortical  response  to  a  pattern  decays  rapidly  (Bienenstock  et  al..  1982).  The 
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final  class  of  problems  concerns  the  evolution  of  synaptic  strengths. 
Physiological  studies  have  yet  to  clarity  which  cell  types  develop  stimulus 
selectivity.  Thus  we  need  to  make  assumptions  about  the  evolution  of 
geniculocortical  synapses  on  inhibitory  and  excititory  cells.  In  addition* 
the  role  of  plasticity  in  intracortical  synapses  is  not  clear. 

The  many-neuron  problem  has  been  treated  before  by  other  researchers. 

Von  der  Malsburg  (19731  chose  an  anatomy  of  two  cell  types  corresponding  to 
excititory  and  inmbitory  cells.  In  his  model*  the  two  cell  types  were 
distributed  equally  but  only  the  excititory  cells  had  geniculocortical 
atferents.  The  problem  of  response  dynamics  was  solved  according  to 
alternative  (a)  above.  Finally*  von  der  Malsburg  chose  an  intracortical 
circuitry  in  which  a  cell  received  static  synapses  that  were  short  range 
excititory  and  long  range  inhibitory. 

Since  this  work,  further  experimental  studies  have  emphasized  the  role  of 
cortical  inhibition  in  neuron  selectivity  (Sillito*  1975).  However*  evidence 
suggests  that  both  inhibitory  and  excititory  cells  receive  geniculate 
afferents  (Davis  and  Sterling,  1979).  Our  model  uses  a  circuitry 
incorporating  excititory  and  inhibitory  cells  which  both  receive  geniculate 
input.  We  asstime  alternative  (b)  above:  Only  monosynaptically  and 
disynaptically  mediated  components  of  the  geniculate  message  are  integrated  to 
give  a  cell's  response.  One  of  the  results  of  incorporating  a  'first  order 
anatomy'  into  the  many-neuron  problem  has  been  a  resolution  of  the  'ideal 
synapse'  into  seperate  excititory  and  inhibitory  synapses.  It  can  be  shown 
that  under  proper  conditions  on  the  intracortical  inhibitory  synapses*  and 
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assuming  a  non-selective  inhibitory  cell*  excititory  cells  that  evolve 
according  to  equation  3b  will  reach  a  state  maximally  selective  with  respect 
to  their  own  individual  environments.  Thus*  with  respect  to  cortico-cortical 
connectivity,  selectivity  appears  to  depend  more  on  inhibition  than  than  it 
does  on  excitation.  Further  work  on  the  precise  spatial  relationships  ana 
evolution  equations  of  the  intracortical  circuitry  is  necessary  before  the 
mechanisms  specifying  global  cortical  properties  will  be  fully  understood. 
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APPENDIX 


The  function  ^(c,£)  that  describes  the  post-synaptic  effect  on  the 

_  b 

evolution  of  the  synaptic  efficacies  has  zeros  at  c  =  0  and  at  c  =  c  .  For 
those  values  of  c,  the  synaptic  strengths  will  not  change.  The  values  of  m  for 
which  <f>-  0  when  d  ^  0.  are  called  fixed  points.  The  fixed  points  of  m-space 
can  be  classified  according  to  the  following  criteria.  A  fixed  point  of  index 
s  is  a  point  in  m-space  for  which  nffd'  =  (nf*3)^l  4®,  with  dJ£  i ’=  {d*  d* } 

and  m*.d‘,=  0  for  d‘6ofi  =  {d**»...«  d*}. 

The  stability  of  a  fixed  point  will  be  investigated,  rather  informally, 
by  examining  the  time  dependance  of  m  if  it  is  perturbed  from  the  fixed  point, 
m*.  A  point  is  unstable  if  sfter  a  small  perturbation  m  =  m*  +  x,  Qxl«l®*t), 
m  evolves  away  from  m*.  A  point  is  stable  if  m  returns  to  m*  after  the 
perturbation. 

Under  proper  restrictions  on  we  can  assume  that  ft  is  a  linear  function  of 
its  argument  for  m  near  a  fixed  point.  Thus  for  c  near  zero,  ^(c,c)a/  -o£,c»  and 
for  c  near  c^,  jKc,c)/v  <X*(c  -  cp).  In  the  special  case  of  c  equal  to  zero,  we 
assume  that  ^(c,0)a/<*  ca.  Where  we  define  ex,  =  and  ** =  • 

These  assumptions  greatly  simplify  the  analysis  of  the  local  stability  of  a 
fixed  point. 
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The  environment  of  the  cell  is  assumed  to  consist  of  K  linearly 
independent  vectors®^  =  {d'.....dK}.  Thus  we  define  the  average  vector: 

n 

d  =  1/K  d*. 

aft 

Then  for  d^S  J&  .  dJ£ 

m*> d1  =  0 

m*>dJ'  -  (m**  d)4-  ; 

With  ~  (K/s)3,  for  s  >  0. 

and  yM,)=  0  for  s  =  0.  Here  we  have  specialized  to  p  =  2. 

The  change  in  the  synaptic  strengths  as  a  result  of  these  vectors 
entering  is: 

m  =  -  oc.tm-d*- )d‘  ,  d*"  €  <& 

m  =  +<X)(t(m*d^  -  (m*d)a  )dJ'»  dJ*€  o&^, 

m  is  very  near  the  fixed  point,  so  that  m  =  m*  +  x.  If  forms  a  complete  set 
(i.e.,  K  =  N)  then  we  can  expand  x  in  the  set  0&"  : 

=  £ 


x 
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Thus  the  evolution  of  the  perturbation  is  given  by  (m  =  x): 


x  =  -  o(,  (x-dl  )d* 


x  =  -o^(x-d)dJ  +  CKx*) 


.  * 
where  d  =  dJ (2/s  -  1)  +  2/s  5"  d* 


x  =  OC  (x*d^)Ad^»  for  i  =  1,2,. ...K. 


S  >  O 


8  =  0 


’  '  .  >» 

Now  x  =  x  d  .  Thus  x  =  £  x^d*.  For  the  K  linearly  independent  vectors, 

4-»l  ■* 

the  coefficient  of  each  vector  must  vanish  separately.  Thus  the  expectation 
value  over  the  environment  yields: 


x.  =  -o^x-d1) 
xj  =  -oi^x-d) 
x^  =  (x-d1)^ 


8  >  0 


8  =  0 


or 


To  illustrate  we  consider  the  simplest  environment:  a  set  of  K  =  N 
thonormal  vectors,  =  {dl,...,  dfy;  Then 


*1*  -«.*l 

xj=  -«^,(2/s-l)xj -o^2/s  Xj, 


"A 

► 


8  >  0 
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x.=  «  (x^)*  8  =  0 

where  £  means  a  sum  over  all  q  such  that 

» 

Ve  see  immediately  that  for  s  =  0,  the  perturbation  x  grows  faster  than 
exponentially*  and  m*  is  an  unstable  fixed  point.  If  s  >  0,  then  x*=  xtt ojft. 
and  those  components  of  the  perturbation  along  the  vectors  for  which  m*.  d  =  0 
will  decay  exponentially.  What  of  the  components  of  x  along  the  'preferred* 
vectors  for  which  m**d  =  (m**d  )4  ?  Solving  the  second  equation  gives  for  s  =  1 
and 


xj(t)  =  (xj(0)~ 


+  o4-o^xe 


And  for  the  special  case  of  : 

xj(t)  =  (xj(0)-2o^Xt)e  *** 

We  have  made  the  definition  that  X  =  21.  xa(0)*  Thus  for  s  =  1»  all 
components  of  the  perturbation  will  vanish  as  t  tends  to  oo  ,  The  s  =  1  point, 
that  point  for  which  c  =  c  4  for  one  pattern  and  c  =  0  for  all  other  patterns* 
is  a  stable  point. 


Consider  now  s=2.  For  this  value  of  s*  two  patterns  give  a  non- zero 
response,  and  all  others  give  a  sero  response.  Then =  {d1 ,  d*"}  and •Br =  id1*..., 
The  solutions  of  the  differential  equations  for  the  components  of  x  along  d' 
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and  d4,  are  (for  <*«  *6  o ^  )  s 

x,(t)  =  ^(x,(0)  -  xA(0))e<'/“t+  i(x,( 0)  +  xA(0)  “«£*£x>e  *t+ 

x4(t)  =  ’4(xx(0)  -  x,(0))eA  +  &(xA(0)  +  x(  (0)  ~  J~^x)e"<*‘t+o^£l e'**** 

and  for  o(0  =  : 

x,  (t)  =  Ji(x,(0)  -  x^  ( 0 ) )  e**  +  /c.(x,  (0)  +  x^O)  -  ^  Xt)e  °l,t, 

xx(t)  =  ,/a.(xA(0)  -  x,  (OJle0*0*  +  ^(x4(0)  +  x,  (0)  -c>4Xt)e  °<et. 


Thus  only  in  the  special  case  of  x,(0)  =  x4(0)  wilt  the  b  =  2  point  be  stable. 
However  this  means  that  if  the  perturbation  is  only  slightly  off  the  line  in 
m-space  on  which  x,=  x4,  then  the  perturbation  will  grow  rapidly.  This  simply 
means  that  the  s  =  2  point  is  a  saddle  point.  Further  analysis  for  s  >  2 
yields  similar  results;  for  s  >  2»  the  fixed  points  are  unstable.  Our  linear 
analysis  indicates  that  for  the  case  of  an  orthonormal  environment »  only  the 
s  =  1  point  is  stable  asymptotically. 

This  form  of  analysis  can  be  carried  out  for  the  general  linearly- 
indepenoent  environment  in  two  dimensions.  The  results  found  above  hold 
for  this  general  environment:  only  the  s  =  1  point  is  stable  asymptotically. 
The  analysis  for  the  non-orthogonal,  K-dimensional  environment  is  currently 
under  investigation. 
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These  same  methods  may  be  used  to  illustrate  the  correlation  between 
ocular  dominance  and  selectivity  in  the  monocular  deprived  environment. 

Following  the  arguments  presented  in  Bienenstock  et  al.  (198^)  we  consider  an  MD 
environment  specified  by  the  circular  environment  dr  for  the  right  eye  ana  n  a 
"pure  noise  vector  for  the  left  eye.  We  now  assume  the  cell  has  evolved  to 
the  state  given  by  ^(m*,  0)  where  m*  is  a  stable  selective  state  in  the 
environment  d,,.  As  before,  we  consider  a  perturbation  from  the  fixed  point; 
the  state  of  the  cell  now  is  given  by  fJ(m£+  x,. ,  xx).  The  perturbation  then 
evolves  according  to  (where  the  noise  is  assumed  to  have  zero  mean): 

xr  =  (m**  dj,  +  x^dr  +  x*n,  m*.  d^  +  )dr 

x*  =  (m*.  dr  +  x^.d^  +  x^.n,  m*.d^.  +  x^d^)n. 

The  stability  of  the  equation  for  x,.  has  already  been  assumed  in  the  conditions  a 
on  m*.  The  equation  for  x^is  analyzed  by  dividing  the  right  eye  inputs  into  those 
for  which  ^(m^dp,  nyd,,)  is  far  from  zero  (either  above  or  below)  and  those  for 
which  ^(m^d,,,  nyd,,  )  =  0.  For  those  vectors  in  the  first  class,  the  sign  of 
is  determined  by  d^, alone,  hence  the  equation  for  x^ is  the  equation  of  a  random  walk 
Those  vectors  in  the  second  class  elicit  a  response  near  the  threshold  0  ,  or 
near  zero.  Using  the  methods  described  above,  it  is  easy  to  see  that  vectors 
with  a  response  near  0  cause  the  perturbation  to  evolve  according  to: 


xA <v  of^,(^.n)n 
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and  those  for  which  mj^dy,*  0  cause  the  perturbation  to  evolve  like: 
oi«(^t.n)n 

where  CC0and  <XMare  detmed  above.  Finally,  averaging  these  equations  (the 
distribution  of  n  is  assumed  symmetric  with  respect  to  x^ )  yields, 
respectively 


\  A'~0,°  nf  xx 

where  n?  is  the  average  squared  magnitude  of  the  noise  input  to  a  single 
synapse  from  the  closed  eye. 

Now  for  the  case  of  K  linearly  independant  vectors  defining  the 
environment  d^,.  only  vectors  of  the  second  class  enter  the  system  after  (m*  ,0) 
has  been  reached.  Also,  for  m*  a  stable  selective  state  (i.e.,  s  =  1),  only  one 
vector  of  dr  will  elicit  a  response  near  threshold.  Thus  the  results  above  show 
that  only  one  input  will  cause  the  perturbation  to  grow  and  all  others  tend  to 
return  the  state  of  the  cell  to  (m*  ,0).  For  the  general  environment,  the  more 
selective  m*  is  with  respect  to  d,,.  the  greater  the  number  of  inputs  that  will 
drive  (m*  +  x^,  x^ )  to  (m*.  0).  This  linear  analysis  then  allows  us  to  conclude 
that  under  proper  conditions  on  and  (dr.  n  ).  (Bienenstock  et  al..  1982),  the 
state  (m*,  .  0)  is  stable  on  the  average  and  the  fluctuation  of  due  to 
'preferred*  patterns  is  smaller  for  more  selective  m*  . 
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FigUTfi  Captions 

Figure  1.  Schematic  representation  of  an  LxN  array  of  ideal  6ynaptic 
junctions.  The  value  gives  the  net  influence  of  the  j-th 
afferent  on  the  i-th  cell. 

Figure  2.  Examples  of  aspecific  (A),  immature  (I),  and  specific  (S) 
response  curves. 

Figure  1.  Distribution  of  the  different  types  of  cells  in  three  age  groups 
in  the  normally  reared  kittens  (upper  part)  and  in  the  dark-reared 
kittens  (lower  part).  The  ordinate  is  normalized  so  that  the  heights 
are  the  percentages  of  cells  in  the  various  function  groups.  Type  1, 
nonactivatable  (|J|{);  2,  nonspecific  (Q);  3,  immature(f^|)  {  4, 

specific  (^).  (From  Fregnac  and  Imbert,  1977 ,  1978.) 

Figure  4»  Evolution  of  the  development  of  the  various  specificity  groups  in 
cats  raised  (&)  normally  and  (b.)  in  total  darkness.  The  results 
are  based  on  an  unweighted  regression  analysis  of  1050  cells  (From 
Fregnac,  1978.) 

Figure  £.  Distribution,  in  percentage,  of  the  three  types  of  visual 

cortical  units  (area  17)  recorded  after  6  hours  of  visual  exposure  for 


6 -week-o Id  dark  reared  kittens.  Columns:  I,  dark-reared  kittens;  IV* 
normally-reared  kittens.  During  6  hours  of  exposure*  conditions  were: 
in  II  and  III*  freely  moving;  in  III*  12  hours  in  the  dark  followed 
the  6  hours  of  exposure.  Numbers  of  visual  cells  recorded  are  given 
under  each  column.  Specific  cells  (|^|)  are  activated  by  oriented 
stimuli  within  a  sharp  angle  (<60°).  Immature  cells  (^jj)  are 
activated  by  oriented  stimuli  within  a  larger  angle  (<150°). 
Nonspecific  cells  (£^|)  are  activated  by  nonoriented  stimuli  moving  in 
any  direction.  A  statistical  analysis  reveals  no  significant 
difference  in  the  percentages  of  immature  and  specific  units  in 
between  columns  III  and  IV.  Therefore  it  may  be  that  for  a  6-week-old 
dark-reared  kitten*  a  6-hour  exposure  to  visual  input  followed  by  12 
hours  in  the  dark  is  sufficient  to  produce  a  distribution  of  cortical 
cells  similar  to  that  of  normally  reared  animals.  (From  Buisseret* 
Gary-Bobo,  and  Imbert*  1978.) 

Figure  fi..  The  modulatory  function  ^(c»0(c))  is  shown  for  two  values  of 

6(c).  As  c  increases*  so  does  6.  The  modification  is  (anti-)  hebbian 
((anti-)  parallel  to  the  input  vector  d)  if  the  evoked  response  c  is 
greater  (less)  than  0. 

Figure  2..  The  state  space  of  a  two-synapse  cell  in  a  two-pattern 

environment  is  shown.  For  clarity,  the  vectors  are  chosen  to  lie 
along  the  axes  (d^=(l*0)  and  d^=(0*l))»  but  this  is  not  necessary 
(see  figure  3  in  Bienenstock  et  al.,  1982).  (a).  The  directions  of 

change  in  the  state  m  as  a  result  of  each  input  (open  arrowheads  for 


<i; -  solid  for  d^)  are  shown  for  each  region  ana  on  the  isoclines 
which  separate  them.  Each  fixed  point  is  labelled  by  its  order  s. 

One  can  see  that  along  the  isoclines,  ^=0  with  respect  to  one  of  the 
patterns.  The  s=2  point  is  a  saddle  point:  it  attracts  trajectories 
from  some  directions,  but  repels  them  in  others,  so  it  is  not  stable. 
The  s=0  point  repels  all  nearby  states.  Hence  only  the  8=1  points  are 
stable,  (h.).  Some  sample  trajectories  are  shown  with  the  isoclines 
(bold). 

Figure  &.  Results  of  machine  simulation  of  neuron  development  in  four 
experimental  paradigms:  (a)  Normal  Rearing,  (fe.)  Binocular 
Deprivation,  (&)  Monocular  Deprivation,  ana  (d.)  Reverse  Suture. 

Each  figure  shows  the  response  vs.  orientation  of  the  test  stimuli  to 
each  eye  both  before  (dashed)  and  after  (solid)  a  period  of  plastic 
development. 
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